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ABSTRACT 

Post-secondary classes are usually followed by mandatory summative evaluations, yet 
changes in teaching and course structure cannot be made with summative evaluations. 
Formative evaluations in online education can result in candid responses from students if a 
third-party facilitator is involved in the process. This paper presents a method for conducting 
formative evaluation in an online class by utilizing the assistance of a colleague. Results of 
one such evaluation are presented, as are its advantages and disadvantages. 
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INTRODUCTION 

In many cases, students complete instructor evaluations at the end of a semester, after 
which it is too late for the instructor to make changes in his/her instruction. Moreover, all 
too often, summative evaluation, a hallmark of formal instructor appraisal, tells us little 
more than how popular we are as faculty. This paper describes a practical method of 
formative online class evaluation involving a third-party "facilitator." The evaluation is 
economical and yields lasting information to assist online instructors in developing the 
quality of his/her teaching. Below, two evaluation cases are presented with the student 
feedback from one of the online classes that was evaluated. 

BACKGROUND 

This method of formative distance education evaluation is rooted in Group Instructional 
Feedback Technique (Angelo & Cross, 1993) and has been modified from the face-to-face 
classroom evaluation techniques of Assessing Learner Reactions to Instruction, the 
Classroom Assessment Technique, and an economical assessment referred to as Barebones 
Questions (Snooks, Neeley, Williamson, Bergman, McEnery, & Young, 2001). Barebones 
Questions in a traditional classroom involves the instructor recruiting a fellow instructor to 
facilitate an evaluation process. The process typically consists of a facilitator entering a 
classroom and presenting three questions to a class: 

> What are one or two specific things your instructor does to help you learn in this 
class? 

> What are one or two specific things your instructor does that hinder or interfere with 
your learning? 

> What are one or two specific, practical suggestions you can give so your instructor 
can improve your learning in this course? 

Group Instructional Feedback in a Face-to-Face Class 
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The facilitator in a face-to-face class is limited to 30 minutes and has the class break into 
small groups to respond to the items. After 15 minutes, the facilitator uses a transparency 
sheet on an overhead projector or a computer with a digital projector to capture and list the 
class responses, noting any additional comments from the students. The facilitator then 
leaves the classroom with the evaluation information in hand. 

Following the classroom portion of the evaluation, the facilitator and the class instructor 
meet for a 30 minute session to "debrief" on the comments of the students, thus giving the 
instructor the opportunity to hear second hand what the students' opinions were and what 
they can offer as suggestions. When this scenario is conducted mid-semester, the instructor 
has ample time to alter his/her instruction to enhance the class. 

Modifying Face-to-Face Group Instructional Feedback Evaluation Techniques in 
Asynchronous Distance Education Similar to the method used in face-to-face classes, an 
instructor in an online class can invite a colleague to facilitate a formative evaluation in a 
distance education environment using the same three questions noted above. The facilitator 
can use any number of methods to contact students electronically to ask the questions. 
Students then respond to the facilitator, who deletes identifying information from students' 
correspondences, then forwards the responses to the instructor. 

The sections below describe how this evaluation technique was implemented in two 
asynchronous distance education classes in the United States. 

Case One— Marketing 5031 Online, Foundations of Marketing was a 14-student, upper- level 
undergraduate class taught via WebCT, text reading, and bulletin board discussion in a large, 
multi-campus university system in and around the Houston, Texas area in the United States. 
The instructor granted the facilitator access to the online class WebCT site. 

The facilitator in this case was a professor at a university located nearly 500 kilometers (300 
miles) away in San Antonio, Texas. The instructor introduced the facilitator to the class via 
e-mail and explained the purpose of the activity. 

The facilitator, on an agreed upon date, submitted the three evaluation questions to the 
online marketing class via WebCT's web-based bulletin board function and explained that 
the students should respond to the questions within a certain time frame. 

The students had the opportunity to see and comment on one another's responses to the 
questions. The facilitator copied the students' responses, eliminating duplicate responses 
and students' names, pasted the comments in an e-mail message, and sent the message to 
the class instructor by e-mail. 

The online class instructor then had the opportunity to ask questions of the facilitator 
regarding the students' responses. 

Case Two— Education 6300 Online, Introduction to Education Technology was a seven- 
student; graduate-level class taught via e-mail discussion group and Web page/online 
readings at a small. United States Catholic liberal arts university in San Antonio, Texas. A 
colleague of the classes' instructor coordinated the evaluation process in this case. 

The instructor granted the facilitator access to the class e-mail discussion group and the 
facilitator e-mailed the three questions to the students in the e-mail discussion group. 
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The students where previously paired by the course's instructor and they were instructed by 
the facilitator to develop, in their small groups, responses to the questions and to reply by e- 
mail directly to the facilitator by a given date. 



Table: 1 

Student responses (n=7) to online formative 
evaluation questions in Introduction to Education Technology 



What are one or two specific things your instructor does that help you learn in this course? 

• My instructor has been helpful whenever I have requested help. 

• He is very punctual in getting feedback to us on our assignments. 

• Well, he is encouraging in all of his communications and quick on the turn-around time 
of our papers. 

• The instructor gives productive suggestions on how to solve problems that occur while 
working on class assignments. 

• He has given me many opportunities to get help. 

• I have had MUCH MORE interface with the instructor because of the online format. 
Additionally, I found the instructor's course very challenging. The instructor's format 
forces me to attempt tasks that I may have prejudged as useless in a classroom and/or 
avoided because of that prejudice. 

What are one or two specific things your instructor does that hinder or interfere with your 
learning? 

• The instructor is very knowledgeable about computers and sometimes would lose me 
when it came to explaining some items, but he was always willing to help me 
understand. 

• There were a few assignments that I misunderstood, but I did clarify them later by 
meeting online with other students in this class. 

• Well, I would have liked us to use the Blackboard [Web-based learning environment] 
applications more. 

• I was looking forward to us building a cyber-classroom on the net. 

• Maybe that we would each take turns monitoring the site? And put it out there for us 
to interact more with each other? 

• Sometimes I get the feeling there are only four or so people in the course. It would be 
nice to *meet* the others, not just the ones who aren't afraid of free falling in 
cyberspace. 

• I would appreciate more than just the reading of words on e-mail to exchange ideas. 

• The course is still in need of some hands-on activities to check for understanding. 

• I would have enjoyed having some lab on how to do a Web page. 

• The instructor does not interfere with learning, there are times, however, that being at 
a distance hinders the learning process. There are times when an introductory class 
needs to be hands-on. 

• I need the hands-on learning and direct teaching method for comprehension. 

• I would have loved to learn the different short cuts to Web page design. 

• I found that I could not answer this question. Not because it was hard or difficult, but 
because it really didn't apply. The courses I've taken have been very enjoyable. The 
format of the course gave me a chance to focus on a project or skill that will help my 
school district and the pupils. 

What are one or two specific, practical suggestions you can give so your instructor can improve 
your learning in this course? 

• I feel that we needed to meet at least once, because I did get confused on what and 
how to do certain assignments. I did enjoy not having to drive all the way into San 
Antonio to attend class, but I did need additional help. It would have helped if we met 
as a group at least once during the course. 

• Suggest a few lab days for those who need the extra tutoring in their assignments. 

• Promote more online asynchronous courses. 

• The instructor's attention-to-detail on the written assignments can be a royal pain, but 

because of his attention-to-detail my professional skills have improved. 
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The facilitator compiled the students' responses, removed student names, eliminated 
duplicative replies, and e-mailed the results back to the entire class via their e-mail 
discussion group. The instructor, being a member of the class e-mail discussion group, 
received the results in the same message in which the students received the results. 

In order to follow up, the class instructor and the facilitator exchanged several clarifying e- 
mail messages after the results were sent to everyone via e-mail. Given mid-semester, the 
instructor was then able to modify his instruction as needed to enhance the class based 
upon student feedback. 

Table: 1 presents the students' responses as compiled by the evaluation facilitator for the 
Education 6300 online class. 

ADVANTAGES AND DISADVANTAGES OF FORMATIVE EVALUATION ONLINE 

Formative evaluation has inherent advantages and disadvantages, some of which are listed 
below: 

Advantages 

> Collegial collaboration breaks the isolation of teaching whereby instructors can see 
how their colleagues conduct classes and can learn from students' responses. 

> Students have the opportunity to be "heard" in a formal way and they know another 
instructor is involved and is "watching" what is going on in the online class. 

> The format of the questions yields constructive criticism rather than negative 
feedback. 

> Online formative evaluation is economical in time, requiring only about 30 minutes 
each for the instructor and the facilitator. 

> In an era where students are increasingly viewed as customers, or clients, instructors 
can modify their courses to best suit the learning needs of their students. 

Disadvantages 

> It takes coordination with another person to conduct such an evaluation. 

> The facilitator must be sufficiently skilled in technology use to be able to conduct the 
evaluation in an online environment. 

> An instructor must have confidence in his/her teaching to share it with a colleague. 

Inherent to formative evaluation online is the need to act upon any responses you receive as 
a class instructor, otherwise, you can appear as if you do not care and the evaluation is 
simply another activity students must complete. 

Advantageous to the instructor, as opposed to summative evaluation, is the fact that if you 
do act upon student input, you are likely to be perceived as a better instructor by your 
students— a fringe benefit of the process. Instructors can also include the results in their 
annual evaluations conducted by their departmental chairs. 

Moreover, regardless of the outcome, students take on a responsible role that encourages 
them to participate and they become aware of broader, class-wide observations and other 
students' perceptions. 

Given the digital nature of distance education, this evaluation technique can be conducted 
completely online, from any location with relative ease and anonymous student feedback 
making this a method for improving distance education. 
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